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WHAT IS CLAIMED IS: 



1 1 . A data processing system comprising: 

2 a bus system; 

3 a CPU connected to the bus system; 

4 a RAM connected to the bus system, the RAM being divided into pages, each 

5 page having an execution flag; 

6 a memory manager configured to manage the pages of the RAM and permit CPU 
S 7 execution of data on pages according to the execution flag; 

V i 8 a program stored within at Least one page of the RAM; and 

if 9 a program stack stored within at least one page the RAM, 

U 1 0 wherein the memory manager is configured to determine whether the program is 

sj 1 1 susceptible to buffer overflow attacks, and, if so, set the execution flag for program stack 

!\ 12 pages of RAM to deny CPU execution of data on the program stack pages of RAM. 

ml 2. The data processing system of claim 1 wherein the memory manager and the CPU 

=; 2 are configured to deny CPU execution of data by triggering a hardware interrupt. 

1 3. The data processing system of claim 1 further comprising: 

2 a process structure table in data communication with the memory manager, 

3 wherein the memory manager comprises an annotation API, 

4 wherein the annotation API is configured to annotate within the process structure 



table the susceptibility of the program to buffer overflow attacks, and 

wherein the memory manager is configured to make the determination of 
susceptibility to buffer overflow attacks with reference to the process structure table. 
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1 4. The data processing system of claim 2 further comprising: 

2 a process structure table in data communication with the memory manager, 

3 wherein the memory manager comprises an annotation API, 

4 wherein the annotation API is configured to annotate within the process structure 

5 table the susceptibility of the program to buffer overflow attacks, and 

6 wherein the memory manager is configured to make the determination of 

7 susceptibility to buffer overflow attacks with reference to the process structure table. 

w 

H 1 5. The data processing system of claim 1 further comprising: 
y 2 a process structure table in data communication with the memory manager, and 

Ly 3 an annotation program in data communication with the process structure table, 

4 wherein the annotation program is configured to annotate within the process 

f 5 structure table the susceptibility of the program to buffer overflow attacks, and 
M= 6 wherein the memory manager is configured to make the determination of 

« 7 susceptibility to buffer overflow attacks with reference to the process structure table. 



1 6. The data processing system of claim 3 wherein the program is configured to call 

2 the annotation API if the program is susceptible to buffer overflow attacks, 
the memory manager is configured to determine susceptibility upon a request to 



3 



4 allocate an additional page of RAM for the program. 
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7. The data processing system of claim 4 wherein the program is configured to call 
the annotation API if the program is susceptible to buffer overflow attacks, 

the memory manager is configured to determine susceptibility upon a request to 
allocate an additional page of RAM for the program. 

8. The data processing system of claim 5 wherein the program is configured to call 
the annotation program if the program is susceptible to buffer overflow attacks, 

the memory manager is configured to determine susceptibility upon a request to 
allocate an additional page of RAM for the program. 
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1 9. A computer program product in a computer-readable medium adapted to prevent 

2 buffer overflow attacks comprising: 

3 a memory manager code comprising a set of codes operable to direct a data 

4 processing system to manage a set of pages within a RAM of the data processing system 

5 and to permit a CPU of the data processing system to execute data on pages according 

6 to an execution flag on each of the set of pages; 

7 an application program code comprising a set of codes operable to direct a data 

8 processing system to request the memory manager code to establish a program stack 
p 9 within at least one page the RAM; and 

C| 10 a susceptibility code comprising a set of codes operable to direct a data 

:f 1 1 processing system to determine whether the application program code is susceptible to 

UJ 1 2 buffer overflow attacks, and, if so, set the execution flag for the program stack pages to 

Ci 13 deny CPU execution of data on the program stack pages. 

h* I 10. The computer program product of claim 9 wherein the memory manager code 

gS 2 further comprises a set of codes operable to direct a data processing system to deny CPU 

y 3 execution of data by triggering a hardware interrupt. 

1 11. The computer program product of claim 9 further comprising: 

2 a process structure table code comprising a set of codes operable to direct a data 

3 processing system to establish and maintain a process structure table code in data 

4 communication with the memory manager code and to annotate within the process 

5 structure table the susceptibility of the application program code to buffer overflow 

6 attacks, 
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wherein the memory manager code further comprises codes operable to direct a 
data processing system to make the determination of susceptibility to buffer overflow 
attacks with reference to the process structure table. 

1 2. The computer program product of claim 10 further comprising: 

a process structure table code comprising a set of codes operable to direct a data 
processing system to establish and maintain a process structure table code in data 
communication with the memory manager code and to annotate within the process 
structure table the susceptibility of the application program code to buffer overflow 
attacks, 

wherein the memory manager code further comprises codes operable to direct a 
data processing system to make the determination of susceptibility to buffer overflow 
attacks with reference to the process structure table. 

13. The computer program product of claim 1 1 wherein the memory manager code 
comprises the process structure table code as an API. 

14. The computer program product of claim 12 wherein the memory manager code 
comprises the process structure table code as an API. 

1 5 The computer program product of claim 1 1 wherein the application program code 
further comprises a set of codes operable to direct a data processing system to call the 
process structure table code the application program code is susceptible to buffer 
overflow attacks, and 
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the memory manager code further comprises a set of codes operable to direct a 
data processing system to determine susceptibility upon receipt of a request to allocate 
an additional page of RAM for the application program code. 

1 6. The computer program product of claim 14 wherein the application program code 
further comprises a set of codes operable to direct a data processing system to call the 
process structure table code the application program code is susceptible to buffer 
overflow attacks, and 

the memory manager code further comprises a set of codes operable to direct a 
data processing system to determine susceptibility upon receipt of a request to allocate 
an additional page of RAM for the application program code. 

1 7. The computer program product of claim 1 3 wherein the application program code 
further comprises a set of codes operable to direct a data processing system to call the 
process structure table code the application program code is susceptible to buffer 
overflow attacks, and 

the memory manager code further comprises a set of codes operable to direct a 
data processing system to determine susceptibility upon receipt of a request to allocate 
an additional page of RAM for the application program code. 
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1 18. A method for handling buffer overflow attacks against an application program 

2 running on a data processing system, having a CPU and a RAM divided into pages, the 

3 method comprising the steps of: 

4 designating an execution flag for each page of RAM allocated to a stack of the 

5 application program; 

6 permitting CPU execution of data on pages of RAM according to the execution 

7 flag; 

8 determining whether the application program is susceptible to buffer overflow 

9 attacks; and 

10 if the application program is susceptible to buffer overflow attacks, setting the 

m 1 1 execution flag as to the pages of RAM allocated to the stack of the application program 

i -j 

^12 to prohibit execution. 

5 

y, 1 19. The method of claim 18 wherein step of permitting CPU execution comprises the 

2 steps of: 

5 3 examining the execution flag on the page of RAM; 

□ 4 if the execution flag is not set, triggering a hardware interrupt; and 

5 otherwise executing the data on the page. 

1 20. The method of claim 18 further comprising the steps of: 

2 establishing a process structure table; 

3 maintaining the process structure table by annotating within the process structure 

4 table the susceptibility of the application program to buffer overflow attacks, 

5 wherein the step of determining susceptibility to buffer overflow attacks is made 

6 with reference to the process structure table. 
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1 21. The method of claim 19 further comprising the steps of: 

2 establishing a process structure table; 

3 maintaining the process structure table by annotating within the process structure 

4 table the susceptibility of the application program to buffer overflow attacks, 

5 wherein the step of determining susceptibility to buffer overflow attacks is made 

6 with reference to the process structure table. 

1 22. The method according to claim 20 wherein the step of maintaining the process 

0 2 structure table is done once at the beginning of execution of the application program, and 
Q 3 the step of determining susceptibility is performed upon each receipt of a request 
= ; 4 to allocate an additional page of RAM for the application program code. 

UJ 

G 1 23. The method according to claim 21 wherein the step of maintaining the process 

: 2 structure table is done once at the beginning of execution of the application program, and 

H= 3 the step of determining susceptibility is performed upon each receipt of a request 

1 i 

4 to allocate an additional page of RAM for the application program code. 

1 24. The method according to claim 22 wherein request to allocate an additional page 

2 of RAM is a request to allocate the additional page of RAM for the stack. 

1 25 . The method according to claim 23 wherein request to allocate an additional page 

2 of RAM is a request to allocate the additional page of RAM for the stack. 
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